Data Wrangling

library(tidyverse)

## ── Attaching packages ─────────────────────────────────────────────────────────── tidyverse 1.3.0 ──

## ✓ ggplot2 3.3.2     ✓ purrr   0.3.4
## ✓ tibble  3.0.3     ✓ dplyr   1.0.2
## ✓ tidyr   1.1.2     ✓ stringr 1.4.0
## ✓ readr   1.3.1     ✓ forcats 0.5.0

## ── Conflicts ────────────────────────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

D1 <- read.csv("video-data.csv", header = TRUE)

D2 <- filter(D1, year == 2018)

Histograms

hist(D2$watch.time)

hist(D2$watch.time, breaks = 100)

hist(D2$watch.time, breaks = 100, ylim = c(0,10))

hist(D2$watch.time, breaks = c(0,5,20,25,35))

## Plots

plot(D1$confusion.points, D1$watch.time)

x <- c(1,3,2,7,6,4,4)
y <- c(2,4,2,3,2,4,3)
table1 <- table(x,y)
barplot(table1)

D3 <- D1 %>% group_by(year) %>% summarise(mean_key = mean(key.points))

## `summarise()` ungrouping output (override with `.groups` argument)

plot(D3$year, D3$mean_key, type = "l", lty = "dashed")

D4 <- filter(D1, stid == 4|stid == 20| stid == 22)
D4 <- droplevels(D4)
boxplot(D4$watch.time~D4$stid, xlab = "Student", ylab = "Watch Time")

## Pairs

D5 <- D1[,c(2,5,6,7)]
pairs(D5)

## Part II

Create a simulated data set containing 100 students, each with a score from 1-100 representing performance in an educational game. The scores should tend to cluster around 75. Also, each student should be given a classification that reflects one of four interest groups: sport, music, nature, literature.

#rnorm(100, 75, 15) creates a random sample with a mean of 75 and standard deviation of 20
#pmax sets a maximum value, pmin sets a minimum value
#round rounds numbers to whole number values
#sample draws a random samples from the groups vector according to a uniform distribution
score <- rnorm(100, 75, 15)
hist(score, breaks = 30)

S1 <- data.frame(score)

library(dplyr)
S1 <- filter(S1, score <= 100)
hist(S1$score)

S2 <- data.frame(rep(100, 100-nrow(S1)))
names(S2) <- "score"
S3 <- bind_rows(S1,S2)

interest <- c("sport", "music", "nature", "literature")
S3$interest <- sample(interest, 100, replace = TRUE)

S3$stid <- seq(1, 100, 1)

Using base R commands, draw a histogram of the scores. Change the breaks in your histogram until you think they best represent your data.

hist(S3$score, breaks = 10)

Create a new variable that groups the scores according to the breaks in your histogram.

#cut() divides the range of scores into intervals and codes the values in scores according to which interval they fall. We use a vector called `letters` as the labels, `letters` is a vector made up of the letters of the alphabet.
label <- letters[1:10]
S3$breaks <- cut(S3$score, breaks = 10, labels = label)

Now using the colorbrewer package (RColorBrewer; http://colorbrewer2.org/#type=sequential&scheme=BuGn&n=3) design a pallette and assign it to the groups in your data on the histogram.

#Let's look at the available palettes in RColorBrewer

#The top section of palettes are sequential, the middle section are qualitative, and the lower section are diverging.
#Make RColorBrewer palette available to R and assign to your bins

#Use named palette in histogram
library(RColorBrewer)
display.brewer.all()

S3$colors <- brewer.pal(10, "Set3")
hist(S3$score, col = S3$colors)

Create a boxplot that visualizes the scores for each interest group and color each interest group a different color.

#Make a vector of the colors from RColorBrewer
interest.col <- brewer.pal(4, "Dark2")
boxplot(score ~ interest, S3, col = interest.col)

Now simulate a new variable that describes the number of logins that students made to the educational game. They should vary from 1-25.

S3$login <- sample(1:25, 100, replace = TRUE)

Plot the relationships between logins and scores. Give the plot a title and color the dots according to interest group.

plot(S3$login, S3$score, col = S3$colors, main = "Student logins vs Scores")

S3$col1 <- ifelse(S3$interest == "music", "red", "green")

R contains several inbuilt data sets, one of these in called AirPassengers. Plot a line graph of the the airline passengers over time using this data set.

AP <- data.frame(AirPassengers)
plot(AirPassengers)

Using another inbuilt data set, iris, plot the relationships between all of the variables in the data set. Which of these relationships is it appropraiet to run a correlation on?

IRI <- data.frame(iris)
plot(iris)

#Petal length by petal width is appropriate to run a correlation.

Part III - Analyzing Swirl

Data

In this repository you will find data describing Swirl activity from the class so far this semester. Please connect RStudio to this repository.

Instructions

Insert a new code block
Create a data frame from the swirl-data.csv file called DF1

The variables are:

course_name - the name of the R course the student attempted
lesson_name - the lesson name
question_number - the question number attempted correct - whether the question was answered correctly
attempt - how many times the student attempted the question
skipped - whether the student skipped the question
datetime - the date and time the student attempted the question
hash - anonymyzed student ID

Create a new data frame that only includes the variables hash, lesson_name and attempt called DF2
Use the group_by function to create a data frame that sums all the attempts for each hash by each lesson_name called DF3

DF1 <- read.csv("swirl-data.csv", header = TRUE)
DF2 <- select(DF1, "hash", "lesson_name", "attempt")
DF3 <- DF2 %>% group_by(hash, lesson_name) %>% summarise(sum_att=sum(attempt))

## `summarise()` regrouping output by 'hash' (override with `.groups` argument)

On a scrap piece of paper draw what you think DF3 would look like if all the lesson names were column names

knitr::include_graphics("q5.jpeg")

Convert DF3 to this format

library(dplyr)
library(tidyverse)
DF3 %>% spread(lesson_name,sum_att)

## Warning: The `x` argument of `as_tibble.matrix()` must have unique column names if `.name_repair` is omitted as of tibble 2.0.0.
## Using compatibility `.name_repair`.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_warnings()` to see where this warning was generated.

## # A tibble: 41 x 33
## # Groups:   hash [41]
##     hash    V1 Base_Plotting_S… `Basic Building… Clustering_Exam…
##    <int> <int>            <int>            <int>            <int>
##  1  2864    NA               NA               29               NA
##  2  4807    NA               NA               49               NA
##  3  6487    NA               NA               25               NA
##  4  8766    NA               NA               NA               NA
##  5 11801    NA               NA               16               NA
##  6 12264    NA               NA               NA               NA
##  7 14748    NA               NA               29               NA
##  8 16365    NA               NA               NA               NA
##  9 20682    NA               NA               NA               NA
## 10 21536    NA               19               NA               14
## # … with 31 more rows, and 28 more variables: `Dates and Times` <int>,
## #   Exploratory_Graphs <int>, Fu <int>, Functions <int>,
## #   Graphics_Devices_in_R <int>, `Grouping and C` <int>, `Grouping and Chaining
## #   w` <int>, `Grouping and Chaining with dplyr` <int>, Hierarchica <int>,
## #   Hierarchical_Clustering <int>, K_Means_Clustering <int>, Lo <int>,
## #   Logic <int>, Looking <int>, `Looking at Data` <int>, Manipulatin <int>,
## #   `Manipulating Data with dplyr` <int>, `Matrices and Data Frames` <int>,
## #   `Missing Values` <int>, Plotting_Systems <int>,
## #   Principles_of_Analytic_Graphs <int>, Subsetti <int>, `Subsetting
## #   Vectors` <int>, `Tidying Data ` <int>, `Tidying Data with tid` <int>,
## #   `Tidying Data with tidyr` <int>, Vectors <int>, `Workspace and Files` <int>

Create a new data frame from DF1 called DF4 that only includes the variables hash, lesson_name and correct

DF4 <- data.frame(DF1$hash, DF1$lesson_name, DF1$correct)

Convert the correct variable so that TRUE is coded as the number 1 and FALSE is coded as 0

DF4$DF1.correct <- ifelse(DF4$DF1.correct == TRUE, 1, 0)

Create a new data frame called DF5 that provides a mean score for each student on each course

DF5c <- select(DF1, "hash","course_name", "correct")
DF5c$correct <- ifelse(DF5c$correct == TRUE, 1, 0)
DF5 <- DF5c %>% group_by(hash, course_name) %>% summarise(score = mean(correct))

## `summarise()` regrouping output by 'hash' (override with `.groups` argument)

Extra credit Convert the datetime variable into month-day-year format and create a new data frame (DF6) that shows the average correct for each day

Finally use the knitr function to generate an html document from your work. Commit, Push and Pull Request your work back to the main branch of the repository. Make sure you include both the .Rmd file and the .html file.

assignment 2

fangqi liu

9/30/2020

Data Wrangling

Histograms

Part III - Analyzing Swirl

Data

Instructions